Learning Words from Images and Speech
نویسندگان
چکیده
This paper explores the possibility to learn a semantically-relevant lexicon from images and speech only. For this, we train a multi-modal neural network working both on image fragments and on speech features, by learning an embedding in which images and content words that co-occur together are close. Making no assumption on the acoustic model, this paper shows promising results on how multi-modality could help word learning.
منابع مشابه
Design and evaluation of validity of an electronic alternative and augmentative communication system for Persian-speaking children
Introduction: Due to the high prevalence of communication disorders, augmentative and alternative communication methods are one the options ahead to solve the problems of these people. Since there are no complex tools for Persian-speaking children with communication disorders, we decided to design communication assistant software for these children that produces sound output. Materials and Meth...
متن کاملL2 Learners’ Lexical Inferencing: Perceptual Learning Style Preferences, Strategy Use, Density of Text, and Parts of Speech as Possible Predictors
This study was intended first to categorize the L2 learners in terms of their learning style preferences and second to investigate if their learning preferences are related to lexical inferencing. Moreover, strategies used for lexical inferencing and text related issues of text density and parts of speech were studied to determine their moderating effects and the best predictors of lexical infe...
متن کاملVisually Grounded Learning of Keyword Prediction from Untranscribed Speech
During language acquisition, infants have the benefit of visual cues to ground spoken language. Robots similarly have access to audio and visual sensors. Recent work has shown that images and spoken captions can be mapped into a meaningful common space, allowing images to be retrieved using speech and vice versa. In this setting of images paired with untranscribed spoken captions, we consider w...
متن کاملThe Effect of Word Meaning on Speech DysFluency in Adults with Developmental Stuttering
Objectives: Stuttering is one of the most prevalent speech and language disorders. Symptomology of stuttering has been surveyed from different aspects such as biological, developmental, environmental, emotional, learning and linguistic. Previous researches in English-speaking people have suggested that some linguistic features such as word meanings may play a role in the frequency of speech non...
متن کاملThe study of relationship between bilingualism and private speech with English learning in elementary school students
Purpose: This study investigated the relationship between gender and bilingualism with English learning among 7-11 years old students. Methodology: 261 students (124 girls and 157 boys) were selected through multi-stage sampling method from elementary school. Employing check list the level of English scores was obtained .The children's private speech obtained from listening to them when they we...
متن کامل